arXiv: 1509.03781 v2 [cs.OH] 15 Mar 2017 


Axiomatization of Inconsistency 
Indicators for Pairwise Comparisons 

W.W. Koczkodaj J.-P. Magnot ^ 

March 16, 2017 


Abstract 

This study proposes revised axioms for defining inconsistency in¬ 
dicators in pairwise comparisons. It is based on the new hndings that 
“PC submatrix cannot have a worse inconsistency indicator than the 
PC matrix containing it” and that there must be a PC submatrix with 
the same inconsistency as the given PC matrix. 

This study also provides better reasoning for the need of normal¬ 
ization. It is a revision of axiomatization by Koczkodaj and Szwarc, 
2014 which proposed axioms expressed informally with some dehcien- 
cies addressed in this study. 


1 Preliminaries 

Pairwise comparisons have a long and colorful history traced to Ramon Lull, 
called in Latin Raimundus (Raymundus) Lullus, who was a Majorcan philoso¬ 
pher and logician. He is also regarded as a pioneer of computation theory 
for his use of binary numbers and elements of logic. Three of Lull’s works, 
related to electoral systems, are based on pairwise comparisons. The exact 
date of the hrst written publication, “Artihtium electionis personarum” is 
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not known but it is probably between 1274 and 1283. However, it is easy 
to envision that the practical use of pairwise comparisons may go back to 
antiquity, as a hunter needed to decide which of two stones may be more 
suited as a hatchet head. In [19], a recent project of national importance 
was presented. 

The inconsistency concept in pairwise comparisons was presented in [11]. 
However, this was a cardinal (count) inconsistency indicator. Counting in¬ 
consistent triads has been replaced by more sophisticated inconsistency in¬ 
dicators (also known as inconsistency indexes or consistency indexes). The 
axiomatization follows principles generally recognized for constructing all ax- 
iomatizations: simplicity, internal consistency, and independence. Certainly, 
the assumption of minimization is implicit as it is difficult to imagine a sys¬ 
tem based on, let us say, 1,000 axioms would be of any practical use. 

This study provides rehned axioms for inconsistency indicators hence in¬ 
consistency measures. The hrst axiomatization of inconsistency in pairwise 
comparisons was proposed in [16] in 2014 and was followed by an independent 
study [4] which was published in 2015. Regretfully, both proposed axiomati- 
zations proved to be somehow dehcient as the revisions indicate. The lack of 
explicit extendibility to higher sizes was brought to the attention of the hrst 
author of [16] by Brunelli (see [2] and [5]). The new version of [4] has been 
posted on arXiv (31 July 2015, see [3]). However, axioms proposed by [4] and 
[3] have two problems. The hrst problem is the mathematical complexity. 
The second problem is the approximation error tolerance (addressed by our 
axiom A.3). An arbitrarily large value (e.g., 1,000,000% or more) is toler¬ 
ated in the input data by at least one inconsistency indicator (Cl proposed 
in [20]) as demonstrated in [16]. This is exactly what a proper inconsistency 
indicator should not permit to occur. In fact, the detection property of the 
approximation error in the input data was our goal in this study and the 
proposed axiom A.3 is formulated to prevent it. 

The elegant “monotonicity” axiom prevents the approximation error aber¬ 
ration in the input data. It is common sense to expect that the inconsistency 
indicator of a PC sumbatrix H of a PC matrix A cannot be smaller than 
the inconsistency of a bigger PC matrix A. In fact, the intuition behind “the 
bigger PC matrix the more inconsistency” is undisputed. 
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2 Deficiencies of the previous axiomatizations 

The axiomatization proposed by Koczkodaj and Szwarc in [16] was incom¬ 
plete. In essence, axioms were not only formulated for a single triad T = 
(x, ?/, z) but also two of possible four cases of the increase and/or decrease of 
X and z have been considered. Two other cases: 

1 . X being increased and z being decreased, 

2 . X decreased with z increased; 
were overlooked. 

The construction of PC matrices of the size n > 3 was not explicitly in¬ 
cluded in the axioms but it has been assumed to be the same as for Koczko¬ 
daj’s inconsistency indicator in [12] and [7]. 

However, the presented two counter-examples in [16] and their mathe¬ 
matical analysis are correct. They show that an approximation error in the 
input data of an arbitrary value is tolerated by an eigenvalue-based (but not 
only) inconsistency indicator. 

3 Axioms not to be included 

In [4, 3], a mathematically elegant axiom is included with ii(A) = It 

is not included in our axiomatization since it is irrelevant. Why? First of all, 
it is not necessary. Fig. 1 is used to illustrate it. 
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Figure 1: The case of two assessors 

A glimpse at Fig. 1 may not reveal a problem. However, a more careful 
examination is needed to expose that it is extremely aberrant case. In re¬ 
alistic terms, such case should never take place since Assessor 1 has exactly 
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opposite assessments of Assessor 2. It is highly unusual that all opposite 
assessments can be made. Even for n = 3, it is at the level of the probability 
of winning the grand prize in most national lotteries. Expecting that such 
strange assessments (and probably assessors) are equally inconsistent may 
be perceived as a mathematical hoax. For n = 7, the ideal ("the opposite") 
mismatch is on 21 elements with real values hence the probability may be 
lower than winning all national lotteries on our planet. 

Both PC matrices in Fig. 1 have the same triads. It seems that such PC 
matrices should have the same inconsistency indicator but there is no need 
to postulate it. For unconvinced readers, who see the beauty in postulating 
ii{A) = let us recall that: 



Although the equality may look mathematically attractive and it may even 
allure impression that the equality takes place, it is indeed inequality. Simi¬ 
larly, the transposition of a PC matrix A has nothing to do with PC matrix 
A and the formulation of any axiom for it would unnecessarily increase their 
number. 

The “continuity” for inconsistency indicators cannot be an axiom. The 
discrete case: “0 or 1” is of a considerable practical importance as a test for 
“IF” statements but not only). It has been in use since 1939, if not earlier. 
Certainly, a discrete inconsistency indicator with three values: {0, 0.5, 1} 
has also practical use. The inconsistency indicator: 


■ ■f _ j 0 for expression < threshold 
I 1 otherwise 

is not continuous but dehnitely useful. Other useful cases of discrete incon¬ 
sistency indicators exist and there is no reason to exclude them. For these 
reasons, postulating the continuity is not a part of our axiomatization. 

4 Axiomatization of the PC matrix inconsis¬ 
tency indicator 

Let us recall from [16] that PC matrix is a square matrix M = [m^j, n x n, 
with real elements such that niij > 0 for every i, j = 1,..., n: 
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mi2 ■ ■ ■ min 
1 ■ ■ ■ m2n 

. . . ? 

_!_ ... X 

Lmi„ m2„ J 

where represent ratios expressing a relative preference of an entity Ei 
over Ej. The entity could be any object, attribute of it or a stimulus. It can 
be abstract and usually there is not a well established measure such as a me¬ 
ter or kilogram. “Public safety” or “environmental friendliness” are examples 
of such attributes or entities to compare in pairs. 

In this study, a square matrix with strictly positive entries as ratios of 
entities (sometimes called “alternatives” in the decision making) is regarded 
as pairwise comparisons (PC) matrix. Ratios often express subjective pref¬ 
erences of one entity over the other. For this reason, equalizing the ratios 
with Ei/Ej is unacceptable since the entities could be, for example, relia¬ 
bility and robustness of software (commonly used in a software development 
process as product attributes). The use of symbol “/” is in the context of 
“related to” (not the division of two numbers). Problems with some popular 
customizations of PCs have been addressed in [15]. 

PC matrix M is called reciprocal if ^ for every i,j = 1,... ,n (in 

such case, mu = 1 for every i = 1,... ,n). 

Without loss of generality, we can assume that the PC matrix with pos¬ 
itive real entries is reciprocal. If it is not, we can easily make it reciprocal 
by the theory presented in [13]. Every non-reciprocal PC matrix A can be 
converted into a reciprocal PC matrix by replacing aij and Oji with geometric 
means of aij and Oji ■ aji) and its reciprocal value -^j==- 

PC matrix M is called consistent (or transitive) if 

mik ■ mkj = mij ( 1 ) 



for every i,j,k = 1 , 2 ,..., n. 

We will refer to equation (1) as a “consistency condition.” While every 
consistent PC matrix is reciprocal, the converse is false in general. If the 
consistency condition does not hold, the PC matrix is inconsistent (or in¬ 
transitive). In several studies, conducted between 1939 and 1961 ([11], [10], 
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[9], [21], the inconsistency in pairwise comparisons was defined and examined 
but often, credits for it are given for [20]. 

Inconsistency in pairwise comparisons occurs due to superfluous input 
data. As demonstrated in [17], only n — 1 pairwise comparisons are needed 
to create the entire PC matrix for n entities while the upper triangle has 
n{n — l)/2 comparisons. Inconsistency is not necessarily “wrong”as it can be 
used to improve the data acquisition. However, there is a real necessity to 
have a “measure” for it. 

Let us introduce some auxiliary definitions and notations. 

Definition 4.1. Given n G N, we define 

Tin) = {{i,i,k) e {l,...,n} : i<j<k]. 

as the set of all PC matrix indexes of all permissible triads in the upper 
triangle. 


Definition 4.2. A PC matrix is consistent if k) G T{n) atj ■ ajk = aik- 
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Figure 2: A PC matrix and its submatrix 


Definition 4.3. Assume n < m, A and B are square matrices of degree n and 
m, respectively. We call A a submatrix of B (Ac B), if 3cr : {1,..., n} —>■ 
{1,... ,m} an injection, such that \/i,j G {1,... ,?t.} 
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The above definition is illustrated by Fig. 2. PC submatrix A of size 3 
x3 has a triad (|, 3) in the upper triangle and a triad with inverted values 

(3, |, |) in the lower triangle. It was extracted from PC matrix A by deleting 
rows and columns with the same number. In our case (see Fig. 2), row and 
column 1 and row 4 as well as row and column 4 are removed from the given 
PC matrix to get a PC submatrix. 

Remark 4.4. Each PC submatrix of a PC matrix is also a PC matrix. 

Let A and R be PC matrices. 

A is a PC matrix of the size n x n, n > 3, 

R is a PC submatrix of A: B G A of the size mx m and 3 < m < n. 

We postulate that each inconsistency indicator ii should satisfy the following 
axioms: 

A.l (CONSISTENCY DETECTION) 

ii{A) = 0 A is consistent. 

A.2 (NORMALIZATION) 

ii{A) e [0,1] 

A.3 (ERROR INTOLERANCE) 

A distance d on exists that it generates the same topology as |.|, 
and 

V<h > 0, 30 > 0, V(a;, y, z) G (M+)^, d{y., xz) > <h ^ ii{x., y,z) > (f 
A.4 (MONOTONICITY) 

B C A ^ ii{B) < ii{A) 

A.5 (COMPARISONS ORDER INVARIANCE) 

ii{A) does not depend on the sequence order of entities when the same 
preferences are used. 

Reasons and comments: 
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A.l is the most indisputable since it is based on the consistency condition 
introduced probably in [11] (if not earlier). It implies that the consis¬ 
tency indicator should be able to detect inconsistency in each triad. A 
similar axiom was introduced in [4], where the existence of any unique 
element representing consistency was postulated. However, it seems 
reasonable to expect an inconsistency indicator to achieve the value of 
0 for a consistent matrix, since consistency is equivalent to the lack of 
inconsistency. 

A.2 (proposed by WWK in 1993) is of considerable importance. It is ad¬ 
dressed as an independent study in [14]. In brief, two triads: (1,2,1) 
and (10,101,10) are evidently different inconsistency indicators, x * z 
is nearly “equal” to y in (10,101,10) while (1, 2,1) is unacceptable since 
1*1^2 while 10*10 = 101 and seems acceptable for most applications. 
However, the distance d{x * z,y) = 1 in both cases. 

The compatibility in different applications is another compelling reason 
for postulating the normalization of inconsistency indicators. Without 
it, we cannot compare inconsistency indicators, not only of different 
PC matrices but of different PC submatrices of the same PC matrix. 

Normalization is as needed for inconsistency as much as it is needed 
for the probability and fuzzy set membership functions. There is no 
mathematical reason why the probability should be normalized but 1 
of 6 is not as convenient as its normalized equivalent value. From the 
mathematical point of view, it adds nothing of great importance but 
gives a point of reference. In common parlance, this is expressed by 
“50-50 chance” for the probability although 128 in 256 could be equally 
right for such expression. It is worth to notice that the relative error 
is a normalized value of the absolute error. The relative (not absolute) 
error is of a great importance in the engineering and most (if not all) 
applied sciences. 

There is yet another important side effect of the normalization. For all 
dehned inconsistency indicators, we should be able to hnd the maxi¬ 
mum value in a natural way. Regretfully, it is not always guaranteed 
for all existing inconsistency indicators. 

A.3 this axiom is to prevent tolerating (ignoring) an error of an arbitrary 
value for n —)■ cx). Evidently, “hiding” inconsistency is easier in a larger 



PC matrix. Expressing this axiom mathematically has not been easy. 
The more complete reasoning for the need of normalization is provided 
in [14] (archived on arxiv.org hence available for everyone to read it). 
Briefly, no triad {x, y, z) with an arbitrarily large relative error dehned 
as: 


_ ^ '^approx 

t 

can be tolerate when the size of a matrix grows to inhnity (without the 
expectations to reach it). In our case, t = y — x * z. 

For example, this triad (1,10,1) has unreasonable large relative error: 
1,000% since the middle value y = 10 is no where close to the multi¬ 
plication of the remaining the: x*y = l*l = l and the relative error 
in this case is 10/(1 * 1). A reasonable person cannot assume 10 as an 
acceptable approximation (or even “guesstimation”) of 1*1. Such ap¬ 
proximation is unacceptable regardless of how deeply it may be hidden 
in a PC matrix. Certainly, 10 can be replaced by a constant c of any 
arbitrarily large value and the error will still vanish for large enough n 
in {x^,x‘^'^ + c,x^) as demonstrated in [14]. 

It is worth noticing that an error of arbitrarily large value is tolerated by 
the eigenvalue-based inconsistency as it was evidenced by two counter¬ 
examples and mathematically proven in [16]. 

A.4 (contributed by [1]) reflects the common sense perception of the in¬ 
consistency. The inconsistency indicator of what “is a part of” should 
not be greater than what this part was taken from. In other words, 
this axiom asserts that by extending the set of compared entities, we 
should expect that the inconsistency of the PC matrix may increase. 
However, the concept of a submatrix has been used since it decreases 
the size of a PC matrix from any n to 3. For n = 3, the PC matrix 
is generated by a single triad (which is the triangle above or below the 
main diagonal). 

A.5 states that changing of the order of the compared entities should not 
change the inconsistency indicator. This axiom was also introduced. 


( 2 ) 
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for the first time, in [4] in the equivalent form: 

ii{PAP'^) = 


for any permutation matrix P. 

Regretfully, it uses PC matrix for specihcation while the primary ele¬ 
ments are entities (or alternatives), hence this version of the axiom is 
not only simpler but more adequate to PCs. Nevertheless, credits for 
its introduction should be given to authors of [4], 

According to [18]: 

One of the most methodological tools, Occam’s celebrated razor, 
is the maxim that it is in vain to do with more what ca be done 
with fewer. 

Evidently, the proposed axiomatization is simpler and has fewer axioms 
that proposed in [3]. 

Remark 4.5. We do not postulate the continuity since it would exclude an 
important (and evidently useful) discrete indicator 0-1 for checking whether 
or not there is inconsistency in a given PC matrix. We also do not consider 
properties A3 and A4 from [f] as axioms, since they seem to be too detailed 
and sophisticated, hence unnatural. 

The controversial issue of ii{A'^) is addressed in a separate section since 
it is too complex to address it in a sentence or two. 

4.1 The proof of the consistency of axioms 

Example 4.6. The inconsistency indicator was introduced in [12] for a PC 
matrix 3x3, extended in [7] to any size and simplihed in [16] to: 

Kii{A) = max f 1 — min ( ^ 

i<j<k y ^ik J ) 

satishes all the hve axioms. Thus, the system of axioms A.1-A.5 is consistent. 

For the case of n = 3 it is illustrated by the following 3D plot for the 
middle value of the triad set to 1.5 on the scale of 1 to 3 which is strongly 
recommended by [8]. The plot in Fig. 3 is a section of plot 4D which is 
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evidently impossible to provide. The middle variable {y) of a triad (x, y, z) 
was set to 1.5 so plot 3D could be produced for illustration purposes (mostly 
to show that Kii is not entirely trivial). For the following PC matrix: 

1 X 

A= ^ 1 
1 1 
- y z 


y 

z 

1 


we have: 


Kii{A) = Kii{x,y, z). 



Figure 3: Kii inconsistency indicator 3D section of plot for y=1.5 


4.2 The proof of the independence of axioms 

In order to prove that axioms A.1-A.5 are independent, we must construct 
the examples of inconsistency indicators which satisfy all the axioms but one. 
For positive numbers x,?/, z put: 


iiT{x,y,z) 


0 , 

x+y-\-z 
x-\-y+z+l ’ 


a xz = y 
otherwise 


For a given n x n PC matrix A, let us dehne: 
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iii^A) 

ii2{A) 

ih{A) 

iii{A) 

iH{.A) 


\iX + Kii{A)) 
2Kii{A) 


max 

B&PC3,BcA 


0 , 



B = 

1 X 

1 1 

y ' 

1 


1 1 

- y z 

1 

1 

if A is consistent 



I max(ij-fc)gr(n) min 


( (^ik ^3 fc 

Y ^ ^ik 


otherwise 


max ^ di]^^ dj]^. 

{i,j,k)&'T{n) 


Theorem 4.7. For each j G {1, 2, 3,4, 5} indicator iij satisfies all the axioms 
A.1-A.5 hut A.j. 


Proof. From the construction of the indicators it is easy to verify that each 
satishes four out of the hve axioms. We will show that Vj G {1,2, 3,4,5} 
indicator iij does not meet A.j. It is obvious for j = 1 and j = 2, since: 

ihiA) > ^ 


and: 


ii2 


Consider the PC matrix: 


1 1 9 
111 


16 

~9 


> 1 . 


A 


1 2 1 

-11 

2 

111 


If we change the order of the hrst two compared entities and use the same 
preferences, it will take the form: 


B 


1 - 1 ' 
2 11 
111 
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A simple calculation shows that: 


4 

iir,{A) = 

while: 

5 

ik{B) = 

hence, does not satisfy A.5. 

Let us consider, the following PC matrix dehned for t > 0: 


At — 


1 tx 
{tx)~^ 1 

y~^ tz~^ 


y 

-1 

1 


t-^z 


Then Kii{At) = Kii{Ai) and since xz = txt we have that d{y,txt ^y) is 
constant in the t—variable. However, 


lim \ty \z p < ^-tx ^ 

t —^~1“CXD t —^~hCXD 


which shows that there exists no number y as in A.3. It follows that does 
not satisfy A.3. 

Finally, consider the PC matrix: 


B 


1 1 

1 1 

1 i 
2 

1 1 


1 1 
2 1 
1 3 


and its submatrix: 



It is straightforward to verify that: 


2 1 
1 3 

i 1 




1 

2 ’ 


while: 

so. Hi does not satisfy A.4. 

□ 
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5 On the structure of the space of PC matrices 


One can wonder whether or not a function exists (as simple as possible), 
dehned on the set of all PC matrices, which equals to 0 for consistent PC 
matrices, and which can somehow differentiate two inconsistent PC matrices. 
We will describe obstructions and related structures. For this, we hrst recall 
the Peano function, which makes a bijection between [0,1] and [0,1]^. Such 
a bijection exists, but does not fulhll any classical intuition on what is a 
surface or a path. In order to fulhll this intuition, one has to consider objects 
equipped with a dimension, which enables safe topology and/or geometry, 
such as manifolds [22]. As a subset of matrices, it is apparent that the set 
PCn of n X n-PC matrices is a submanifold of M„(]R). Moreover, this is 
well-known that the PC-matrix 


M 


1 

1 

mi2 


mi2 ■■■ min 
1 ■ ■ ■ m2n 


Lmin m 2 n 


1 


is consistent if and only if there is a family (Ai,..., An) € such that: 


mij = 


(M^)” is itself a Lie group for componentwise multiplication. There is a left 
smooth action: 

(M;)" X PCn ^ PCn 

dehned by: 




■ 1 

1 

mi2 ■ ■ 

1 

■ min 

■ m2n 


1 

A 2 

mi2\i 

A 2 

1 

Ulln Al 

An 

Rl2nA2 

■ 

.., An). 

mi2 

A. 

= 

mi2Ai 

An 



1 

1 

1 


An 

An 

1 




m2n 

A. 


- ^In Al 

'm2n^2 


Remark 5.1. This action can be realized the following way. If the matrix: 


1 mi2 

P- 1 


min 

m2n 


Lmin m2n 


1 
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compares data {di,...dn) such that 


di = mijdj, 


then the matrix: 


1 

A 2 

mi2Ai 


mi2Ai 

A 2 

1 


L An Art . . 

minAi W-2nA2 

compares the scaled datas {Xidi,Xndn). 


mi„Xi 

Xyi 

m2nX2 


1 


Under this action, consistent PC matrices appear as the orbit of the "unit 
matrix" 

'l 1 ■ ■ ■ l' 

1 1 ■■■ 1 


1 1 ■■■ 1 


but this action does not identify with one of its orbits, e.g. with 

consistent PC matrices. For this reason, we need to pay more attention to 
the structure of PCn in order to describe which inconsistency indicators we 
get. 


5.1 Inconsistency in PC 3 

For PC matrix of the size 3 by 3: 


M 


"1 

1 

X 

1 

-y 


X y 
1 

1 1 


M is consistent if and only ii y = xz. Evidently, PC 3 is a 3-dimensional 
manifold because it can be parametrized by the 3 parameters x, y and 2 :, 
where as the set CPC 3 of consistent PC matrices is parametrized only by 
the two parameters x and ; 2 , which shows that CPC 3 is of dimension 2. 
Trying to characterize inconsistency is then reduced to comparing xz and y, 
which can be made elementarily by mostly three ways: 


1. evaluate x?/ — z G M, and compare it to 0, 
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2. or evaluate ^ G and compare it to 1 , 

3. or evaluate log a: + log;^ — logy G M, and compare it to 0. 

Theorem 5.2. The maps {x, y, z) ^ ^ and {x, y, z) h - )■ log x + log 2 : — log y 
are invariant under the action of 

Proof. Let (Ai,A 2 ,A 3 ) G 


T 

X 

y 


1 

X 

1 

z 

= 

1 

-y 

1 

z 

1 



1 

xXi 

yAi 

X 2 

A 3 

X 2 

1 

zX2 

xXi 

A 3 



1 

y^i 

2 A 2 


then: 

xXi z\2 

y>'i 7 / ’ 

As ^ 

which proves invariance of {x,y,z) h-)■ Since log a; + log;^ — logy = 
log j , we get the invariance of (a;, y, z) h-)■ log x + log 2 : — log y. □ 

Theorem 5.3. The map {x, y,z) ^ xz — y is not invariant under the action 
o/(m;)3. 

Proof. Let a; = 1, y = 2 and z = 3, and let Ai = 1, A 2 = 2, and A 3 = 3. Then 


112 ill" 

(1,2,3). 113 = 212 


i i 1 

.2 3 . 


3 1 1 

.2 2 . 


then |x2-| = |^l = lx3-2. □ 

Consequently, in order to describe set-theoric structures of inconsistency 
in PC's, the map; 


(a;,y,x) ^ 


1 a; a:x 

A 1 X , log a; + log X - log y 


_!_ 1 

-XZ z 


or the map: 


(a;,y,x) ^ 


1 a; a;x 

-lx, — 
j_ 1 1 y 

.XZ z 


which identify by bijection PC's with, respectively, CPC 3 x M and CPC 3 x 
appear as very natnral. 
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Remark 5.4. These two identifications are related by the bijection: 


exp 


-)■ 


Consequently, we can work indistinguishably with one or another. 

Let us now consider axiom A.5 which will give restrictions on the possible 
invariant inconsistency indicators. The permutation group 03 has 
two generators: 


"0 

1 

o' 


■f 

0 

O' 

1 

0 

0 

and 

0 

0 

1 

0 

0 

1 


0 

1 

0 


so that the invariance under 03 is given by the invariance under these two 
permutations. The PC-matrix 


1 X y 
z 
1 


i 1 


Ly 


is transformed into: 


1 - - 


ri i n 

a; y 


z y 

X 1 - 

and 

z 1 - 

z 


X 

y ^ 1. 


y X 1 


which corresponds both to change logx+log z—\ogy to — (logx+log z—\ogy). 
Consequently, we can give the following class of inconsistency indicators on 
PCs: 


Theorem 5.5. Let / : M —)■ [0; 1] such that 

1. the function f is non decreasing on IR+. 

2. Vx G M, f{x) = 0 a; = 0 

3. 'ix G M, f{—x) = f{x) 


Then 


ii{x, y, z) = /(log X + log 2 : 
defines an inconsistency indicator on PCs- 


logy) 
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5.2 From PC's to PC^ 

In the last section, we have mostly discussed axioms Al, A 2 and A5 which 
somewhat "generate" inconsistency indicators on PC's, where as A.3 and A.4 
enable to extend them form PC's to PCn- 

Axiom A.3 enables to reduce, by induction, inconsistency on PCn to 

inconsistency on PC's- In a PP^-matrix M, there exists 

3x3 PC-submatrices of M. Axioms A.3, A.4 and A.5 imply that, hxing an 
inconsistency indicator on PP 3 , a possible choice is 

ii{M) = sup {p 3 (P)|P C A, P G PP 3 } . 

6 Testing cases 

In this section, we will examine a few testing cases of inconsistency indicators 
to illustrate the relevance of the axioms. 

For single triad generating 3x3 PC matrix, we used iiT{x,y, z). It is ex¬ 
tended for a PC matrix of the n > 3 size by: 

ii{A) := max 0 *^, a^fc). 

{i,j,k)&T{n) 

Example 6.1. Consider the following inconsistency indicators for testing 
our axioms: 

^ ^ _ ^^^^2^nx+lnz-lny 2lnS'-lna;-ln2^ 

2 -« = 1 = e (0; 11 

0 ^ _ j^jj^^glnx+ln2-ln?; ^Inx+ln z-lny-j 

It is an easy task to verify that all the inconsistency indicators ii induced 
by the above triad inconsistency indicators follow all axioms. 

Notice that the last item on the above list is the previously dehned in¬ 
consistency indicator Kii. 

We will provide examples which do not fulhll A.5 

1. ii = l- 


( ^ 1 = r^! 

1 3 ) 3!(n-3) 


18 



2 . ii = 1 — min(e*^ 

3. II = iti for/ce (0;1], 

The next example is a bit strange and provided for curiosity reason. Tech¬ 
nically, it should not be even analyzed here. It is eigenvalue-based inconsis¬ 
tency indicator introduced in [20] as Cl (for consistency index). It was 
proven in [16] that Cl fails the approximation error test expectations so it 
is incorrect. However, it is still in use, hence its analysis will be conducted 
here. 

Example 6.2. The consistency indicator dehned in [20] as CI{A) = , 

where \max is the principle eigenvalue of n x n PC matrix A. 

Proposition 6.3. Cl does not meet axioms: A.2 and A.C 

Proof. To prove the hrst two statements consider PC matrices: 



and all its 3 x 3 PC submatrices: 

' I 2 y 

Bi= i 1 2 

. 2 i 1 . 

1 \ 2' 

2 12 

1 i 1 

2 2 J 

■ 1 2 2 ■ 

^3= I ^ I 

J 2 1 _ 

The principal eigenvalues of A and its PC submatrices are, respectively, equal 
to: 

~ 4.64474, 

Xbi ~ 3.5, Afjj ~ Afjg = 3.05362. 
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Thus, 


CI{A) ~ 0.215, 


while: 

CI{Bi) = 0.25 

and: 

CI{B2) = CliBs) = 0.02681, 
which contradicts axiom A.4. 


Finally, for this PC matrix: 


C = 


1 

9 


U 

its principal eigenvalue is about 10.111 


9 

9 1 J 

hence: 


CI{C) ~ 3.555 

This, obviously, proves that Cl is not normalized hence violates A.2. 

□ 


7 Conclusions 

The proposed axioms are easy to comprehend and simple to use for dehn- 
ing new inconsistency indicators. Absolutely indispensable yet simple axioms 
are: A.3 and A.4. A.3 is to prevent error aberrations. A.4 recognizes the sim¬ 
ple fact that more “uncertainties”hence inconsistencies may exist in a larger 
PC matrix and it is more of a “fact of life” then for discussion. 

Axiom A.l has been known since 1939 (if not earlier). It has never been 
questioned and it is as the consistency condition. A.5 is based on common 
knowledge and universally accepted. Axiom A.2 calls for normalization which 
is (more or less) commonly used in all known and widely used uncertainty 
measures, such as probability, belief functions, or fuzzy membership func¬ 
tions. It has been well presented in [14] both as a mathematical theory and 
practical necessity. 

As all other similar axiomatizations, there is a chance that this may 
be still be perfected in time. This study may not be perfect but it keeps 
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progressing toward the very important goal: the dehnition of useful and 
mathematically sound inconsistency indicators. As the dates indicate, the 
rehnement of this revised version has taken approximately two years. It may 
stand as a testimony to the level of challenges faced in this study. 
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Abstract 

This study proposes axioms for inconsistency indicators in pairwise 
comparisons. The new observation (by Szybowski), that “no PC sub¬ 
matrix may have a worse inconsistency indicator than the given PC 
matrix” is an essential simplihcation of the axiomatization. The goal 
of formulating axioms for all future dehnitions of new inconsistency 
indicators is difficult and as illusive as the inconsistency concept itself. 
This study improves the axiomatization proposed by Koczkodaj and 
Szwarc in 2014. As a side product, the new axiom allows to prevent 
approximation error aberrations of an arbitrarily large value in the 
input data. 


1 Preliminaries 

Pairwise comparisons have a long and colourful history traced to Ramon 
Lull, who was a Majorcan writer and philosopher, and logician. He is also 
considered as a pioneer of computation theory. Three of Lull’s works deal 
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with electoral systems based on pairwise comparisons. The exact date of the 
hrst text, “Artihtium electionis personarum” is not known but is probably 
between 1274 and 1283. However, it is easy to envision that the informal use 
of pairwise comparisons may go back antiquity since a hunter often needed 
to make a decision which of two stones may be better for a hatchet head. 

The inconsistency concept in pairwise comparisons is in [6] but it is a 
cardinal (count) version. The inconsistent triads are counted in various ways. 
It should be stressed that in this study we do not attempt to provide an 
axiomatization for pairwise comparisons or even for the inconsistency. We 
provide axioms for inconsistency indicators. It is important to assume that 
assessments of experts are processed by allowing them to be altered after 
inconsistency is computed and localized. 

Our searches indicate that the hrst axiomatization of inconsistency in 
pairwise comparisons was proposed in [9] and was followed by [3]. Regretfully, 
both need improvement. The lack of explicit extendibility to higher sizes was 
brought to the attention of the hrst author of [9] in [1]. The new version of [3] 
has been posted on arXiv (31 July 2015, see [2]). However, axioms proposed 
by [3] and [2] endure two problems: uncalled-for mathematical complexity 
and lack of prevention of an approximation error of an arbitrary error (e.g., 
10,000,000% or more)in the input. This is not what a proper inconsistency 
indicator should allow its user to enter. In fact, the detection property of the 
approximation error in the input was our goal. Two practical reasons against 
the implicit inclusion of it as an axiom are: 

• the approximation error tolerance still seems to be a controversial issue, 

• it is not easy to express in simple mathematical terms. 

However, the elegant “monotonicity” axiom prevents the approximation 
error aberration in the input data. It is common sense to expect that a the 
inconsistency indicator of a PC sumbatrix H of a PC matrix A cannot be 
smaller that the inconsistency of a bigger PC matrix A. In fact intuition be¬ 
hind the bigger PC matrix the more inconsistency is undisputed. However, 
it also helps to support it with the mathematics. The number of matrix 

elements is while the number of triads is ^ ^ ^ hence the number of 

elements in a matrix size is O(n^) while the number of triads is O(n^). It 
goes without saying that by increasing size of the given matrix, the number 
of triads increases and with it, the possibility for having triads that can be 
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more inconsistent than any of the triads in the given matrix. Inconsistency 
indicator construction can be axiomatized constructively by taking natural 
concepts of the submatrix and the measure of the departure from the consis¬ 
tency condition in a triad by the rules of what resembles a metric notion in 
mathematics. 


2 Axiomatization of the matrix inconsistency 
indicator 

Let us introduce some auxiliary dehnitions and notations. 

Definition 2.1. For n G N, we call a n x n matrix A a pairwise comparisons 
matrix {PC matrix), if Vi, j G {1,... ,n} aij expresses ratios of entities (or 
alternatives) Ei, E 2 ,..., En by positive real numbers. 

Comment: it is not our goal to dehne PC matrix very precisely; it is enough 
to point out that it is a matrix with ratios of entities (sometimes called 
“alternatives” in the decision making) expressing often subjective preferences 
of one entity over the other. However, equalizing the ratios with Ei/Ej is 
not allowed since one of the entities could be, for example, reliability and 
robustness of software development process as its attributes. 

Definition 2.2. For n G N, we call a PC matrix A reciprocal, if Vi,j G 

{!,...,n} a,i = 


Without loss of generality, we can assume that every PC matrix with 
positive real entries is reciprocal since [8] shows that every general PC ma¬ 
trix A can be converted to a reciprocal PC matrix by replacing Oij and aji 
with geometric means of and Oji (which is ■ Oji) and its reciprocal 
value —==. For this reason, we will be using the term PC matrix for the 

y^ij '^ji 

reciprocal case. 

Definition 2.3. Given n G N, we define 

Tin) = {{i,j,k) G {!,...,n} : i<j<k}. 
as the set of all PC matrix indexes of all possible triads in the upper triangle. 
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Definition 2.4. A PC matrix is consistent if V(i, j, k) G T(n) Oij -Ojk = a^k- 

Inconsistency in pairwise comparisons occurs due to superfluous input 
data. As demonstrated in [10], only n — 1 pairwise comparisons are needed 
to create the entire PC matrix for n entities while the upper triangle has 
n{n — l)/2 comparisons and not all of them may be consistent with others. 
It is not necessarily “wrong” to have inconsistency since it can be used to 
improve the data acquisition but it is a definitely real necessity to have a 
“measure” of it. 

Definition 2.5. Assume n < m, A and B are square matrices of degree n and 
m, respectively. We call A a submatrix of B (Ac B), if da : {1,..., n} —)■ 
{1,... ,m} an increasing injection, such that \/i,j G {1,... ,n} 

dij = ba(i)a{j)- 

The above definition is illustrated by Fig. 1. The PC matrix 3 x 3 is 
generated by triad (|, |,3) and has a triad with its inverted values (3, |, |) 
so we need to delete all rows and all columns not containing these values. 
They happen to be rows and columns 1 and 4 (always of identical number 
for a reciprocal PC matrix). 

Remark 2.6. Each submatrix of a PC matrix is also a PC matrix. 


4 



Let A and B be PC matrices. 

Each inconsistency indicator ii should satisfy the following set of axioms: 

A.l (CONSISTENCY DETECTION) 

n(A) = 0 -v^ A is consistent. 

A.2 (COMPARISONS ORDER INVARIANCE) 

n(A) does not depend on the sequence order of entities when the same 
preferences are used. 

A.3 (LOCALITY) 

[V(q j, k) G T(n) {hk < ttik < aij ■ ajk < ■ hjk) or 

(bik Y Ojfc Y ■ cijj^ Y bij ■ bjk)^ ff(A) Y ii(^B') 

A.4 (MONOTONICITY) 

A C B ^ ii{A) < ii{B) 

A.5 (NORMALIZATION) 

ii{A) e [0,1] 


Reasons and comments: 


A.l is the most indisputable since it is based on the consistency condition 
introduced probably in [6] (if not earlier). It implies that the consis¬ 
tency indicator should be able to detect inconsistency in each triad. 
Similar axiom was introduced in [3], where the existence of any unique 
element representing consistency was postulated. However, it seems 
reasonable to expect an inconsistency indicator to achieve value of 0 
for a consistent matrix, since consistency is equivalent to the lack of 
inconsistency. 

A.2 states that changes of the order of the entities (their permutation) 
should not change the inconsistency indicator. This axiom was also 
introduced, for the first time, in [3] in the equivalent form: 

ii{PAP^) = ii{A), 
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for any permutation matrix P. 

Regretfully, it uses PC matrix for specification while the primary ele¬ 
ments are entities (or alternatives), hence this version of the axiom is 
not only simpler but more adequate to PC method. However, credits 
for its introduction should be given to authors of [3]. 

A.3 (formulated by JS) is crucial for any axiomatization. It represents a 
reasonable expectation that the worsening of all triads cannot make 
the entire matrix more consistent. At the same time, if we improve the 
triads, the inconsistency level should be reduced. 

A.4 (formulated by JS) reflects the common sense perception of the incon¬ 
sistency. The inconsistency indicator of what “is a part of” should not 
be greater than of what it was taken from. In other words, this axiom 
asserts that by extending the set of compared entities, we should expect 
that the inconsistency of the PC matrix may increase. However, the 
concept of a submatrix has been used since it decreases the size of a PC 
matrix from any n to 3 for which is generated by a single triad hence 
we can easily dehne the inconsistency indicator as a triad deviation. 

A.5 (formulated by WWK) was introduced for compatibility purpose in 
applications. It is modeled by the probability and fuzzy set membership 
functions. From the mathematical point of view, it adds nothing of 
great importance but it gives a point of reference. In common parlance, 
this is expressed by “50-50 chance” for probability although 128 in 256 
could be equally right. However, there is an important side effect of the 
normalization. For a well dehned inconsistency indicator, we should be 
able to hnd the maximum value in a natural way. Regretfully, it is not 
always guaranteed for all existing inconsistency indicators. 

The optionality of A.5 comes from a well known fact that every positive 
indicator d can be normalized by a simple mapping d/{l + d) but its 
practical importance compels us to add it to our axiomatization. In 
fact, the automatic normalization by d/{l -|- d) may not always be the 
best solution and in case of Cl it was a problem. 


6 



Remark 2.7. We do not postulate the continuity since it would exclude an 
important (and evidently useful) discrete indicator 0-1 for checking whether 
or not there is inconsistency in a given PC matrix. We also do not consider 
properties Ai and A4 from [3] as axioms, since they seem to be too detailed 
and sophisticated, hence unnatural. 

The controversial issue of iilA^) is addressed in a separate section since 
it is too complex to address it by a sentence or two. 

Example 2.8. The Koczkodaj’s inconsistency indicator: 

Kii{A) = max f 1 — min ( 

y Ojik 

(defined in [7] and enhanced in [4]) satisfies all the five axioms. Thus, Kii is 
consistent with the proposed system of axioms A.1-A.5. 

Kii for the case of n = 3 is illustrated by 3D plot in Fig. 2 for the middle 
value of the triad set to 1.5 and the scale of 1 to 3 strongly recommended in 
[5]. 

The plot in Fig. 2 is a section of a plot4D which is evidently impossible 
to provide. The middle variable (y) of a triad {x,y,z) was set to 1.5 so a 
plot3D could be produced for illustration purposes (mostly to show that Kii 
is not entirely trivial). 

Example 2.9. Consider the consistency indicator defined in [11] as CI{A) = 
where Amax is the principle eigenvalue of A, and n = dim(A). Ev¬ 
idently, it satisfies axioms A.l, and A.2 but the satisfaction of A.3 is not 
evident. The intuition does not support the satisfaction of A.3 hence prob¬ 
ably a counter example may be provided with the next version (the issue is 
still under investigation). However, it does not matter, since the following 
Proposition 2.10 provides a proof that A.4 and A.5 are not satisfied by Cl. 

Proposition 2.10. Cl meets neither A.f, nor A.5. 

Proof. To prove the first statement consider matrices 



and 



7 



B = 


12^2 

- 1 2 - 

2 I 1 2 

- 2 - 1 

2 ^ 2 


The principal eigenvalues for A and B are, respectively, equal to 

Aa = 3.5 


and 


Ar ~ 4.64474. 


Thus, their Saaty inconsistency indices are equal to 

CI{A) = 0.25 

and 

CI{B) ~ 0.215, 

which contradicts the monotonicity axiom. 

Finally, for the matrix 


C = 


1 1 1000 
1 1 1 
1 1 


1 

1000 


its principal eigenvalue is about 11.1, so its Saaty indicator 

CI{C) ~ 4.0. 

This, obviously, proves that Cl is not normalized. 


□ 


3 What went wrong with the axiomatization of 
Koczkodaj / Szwarc? 

First of all, for a triad T = {x,y, z), only two of four cases have been consid¬ 
ered: the increase or decrease of x and ; 2 . Two cases of: 


1. X being increased and being decreased 



2 . X decreased with increased 
were overlooked. 

Secondly, the construction of PC matrices of the size n > 3 was not 
included in the axioms but assumed to be the same as for Koczkodaj’s in¬ 
consistency indicator and it narrows the number of inconsistency indicators. 

However, two counter-examples are correct. They show that something 
may go drastically wrong (tolerance of an approximation error in the input 
data of an arbitrary value) if an erroneous inconsistency indicator is used. 

4 Axioms not to be included 

In [3, 2], a mathematically elegant axiom is included with 

tt{A) = (1) 

It is not included in our axiomatization since it is irrelevant. Why? First of 
all, it is not needed as the attached Fig. 3 presents. 

It seems that there is no problem but a bit careful examination reveals 
that it is the case which, in realistic terms, should never take place since 
Assessor 1 has exactly opposite assessments when compared with the same 
assessments of Assessor 2. It is highly unusual that six strangely ideally 
opposite assessments are at the level of winning a grad prize in most national 
lotteries (and realistic explanations leading to a possible “doctoring” data). 
Expecting that such strange assessments (and probably assessors) are equally 
inconsistent may be perceived as a mathematical hoax. 

Both PC matrices in Fig. 3 have the same triads. By (1), such matrices 
should have the same inconsistency indicator. For unconvinced readers who 
see the beauty in postulating ii{A) = ii{A'^), let us recall that 



although it also looks nice and tempting to believe that the equality could 
take place. Simply, the trasposition of a PC matrix A has nothing to do with 
PC matrix A and the formulation of any axiom for it would unnecessarily 
increase their number. 

The exclusion of the axiom for the approximation error aberration was 
previously discussed with two reasons given. 
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5 Conclusions 


Finally, the proper axiomatization has been proposed. It follows principles 
generally recognized for constructing all axioniatizations: simplicity, internal 
consistency, and independence. Certainly, the minimization is implicit since 
it is hard to imagine a system based on 10,000 axioms. 

The proposed axioms are simple enough for most researchers who use 
pairwise comparisons to comprehend them for dehning their own inconsis¬ 
tency indicators and the use in their applications. 

This is a report of a project in progress and there is still much to be done. 
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Figure 1: A PC matrix and its submatrix 



Figure 2: Kii inconsistency indicator 3D section of plot for y=1.5 
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Figure 3: The case of two assessors 
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